End-to-end Projector Photometric Compensation

Projector photometric compensation aims to modify a projector input image such that it can compensate for disturbance from the appearance of projection surface. In this paper, for the first time, we formulate the compensation problem as an end-to-end learning problem and propose a convolutional neural network, named CompenNet, to implicitly learn the complex compensation function. CompenNet consists of a UNet-like backbone network and an autoencoder subnet. Such architecture encourages rich multi-level interactions between the camera-captured projection surface image and the input image, and thus captures both photometric and environment information of the projection surface. In addition, the visual details and interaction information are carried to deeper layers along the multi-level skip convolution layers. The architecture is of particular importance for the projector compensation task, for which only a small training dataset is allowed in practice. Another contribution we make is a novel evaluation benchmark, which is independent of system setup and thus quantitatively verifiable. Such benchmark is not previously available, to our best knowledge, due to the fact that conventional evaluation requests the hardware system to actually project the final results. Our key idea, motivated from our end-to-end problem formulation, is to use a reasonable surrogate to avoid such projection process so as to be setup-independent. Our method is evaluated carefully on the benchmark, and the results show that our end-to-end learning solution outperforms state-of-the-arts both qualitatively and quantitatively by a significant margin.Comment: To appear in the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Source code and dataset are available at https://github.com/BingyaoHuang/compenne

CompenNet++: End-to-end Full Projector Compensation

Full projector compensation aims to modify a projector input image such that it can compensate for both geometric and photometric disturbance of the projection surface. Traditional methods usually solve the two parts separately, although they are known to correlate with each other. In this paper, we propose the first end-to-end solution, named CompenNet++, to solve the two problems jointly. Our work non-trivially extends CompenNet, which was recently proposed for photometric compensation with promising performance. First, we propose a novel geometric correction subnet, which is designed with a cascaded coarse-to-fine structure to learn the sampling grid directly from photometric sampling images. Second, by concatenating the geometric correction subset with CompenNet, CompenNet++ accomplishes full projector compensation and is end-to-end trainable. Third, after training, we significantly simplify both geometric and photometric compensation parts, and hence largely improves the running time efficiency. Moreover, we construct the first setup-independent full compensation benchmark to facilitate the study on this topic. In our thorough experiments, our method shows clear advantages over previous arts with promising compensation quality and meanwhile being practically convenient.Comment: To appear in ICCV 2019. High-res supplementary material: https://www3.cs.stonybrook.edu/~hling/publication/CompenNet++_sup-high-res.pdf. Code: https://github.com/BingyaoHuang/CompenNet-plusplu

Modeling Deep Learning Based Privacy Attacks on Physical Mail

Mail privacy protection aims to prevent unauthorized access to hidden content within an envelope since normal paper envelopes are not as safe as we think. In this paper, for the first time, we show that with a well designed deep learning model, the hidden content may be largely recovered without opening the envelope. We start by modeling deep learning-based privacy attacks on physical mail content as learning the mapping from the camera-captured envelope front face image to the hidden content, then we explicitly model the mapping as a combination of perspective transformation, image dehazing and denoising using a deep convolutional neural network, named Neural-STE (See-Through-Envelope). We show experimentally that hidden content details, such as texture and image structure, can be clearly recovered. Finally, our formulation and model allow us to design envelopes that can counter deep learning-based privacy attacks on physical mail.Comment: Source code: https://github.com/BingyaoHuang/Neural-ST

Baiji genomes reveal low genetic variability and new insights into secondary aquatic adaptations

The baiji, or Yangtze River dolphin (Lipotes vexillifer), is a flagship species for the conservation of aquatic animals and ecosystems in the Yangtze River of China; however, this species has now been recognized as functionally extinct. Here we report a high-quality draft genome and three re-sequenced genomes of L. vexillifer using Illumina short-read sequencing technology. Comparative genomic analyses reveal that cetaceans have a slow molecular clock and molecular adaptations to their aquatic lifestyle. We also find a significantly lower number of heterozygous single nucleotide polymorphisms in the baiji compared to all other mammalian genomes reported thus far. A reconstruction of the demographic history of the baiji indicates that a bottleneck occurred near the end of the last deglaciation, a time coinciding with a rapid decrease in temperature and the rise of eustatic sea level

A markerless augmented reality system using one-shot structured light

Augmented reality (AR) is a technology that superimposes computer-generated 3D and/or 2D information on the user’s view of a surrounding environment in real-time, enhancing the user’s perception of the real world. Regardless of the field for which the application is applied, or its primary purpose in the scene, many AR pipelines share what might be thinned down to two specific goals, the first being range-finding the environment (whether this be in knowing a depth, precise 3D coordinates, or a camera pose estimation), and the second being registration and tracking of the 3D environment, such that an environment moving with respect to the camera can be followed. Both range-finding and tracking can be done using a black and white fiducial marker (i.e., marker-based AR) or some known parameters about the scene (i.e., markerless AR) in order to triangulate corresponding points. To meet users’ needs and demand, range-finding, registration and tracking must follow certain standards in terms of speed, flexibility, robustness and portability. In the past few decades, AR has been well studied and developed to be robust and fast enough for realtime applications. However, most of them are limited to certain environment or require a complicated offline training. With the advancement of mobile technology, users expect AR to be more flexible and portable that can be applied in any uncertain environment. Based on these remarks, this study focuses on markerless AR in mobile applications and proposes an AR system using one-shot structured light (SL). The markerless AR system is validated in terms of its real time performance and ease of use in unknown scenes

Phycocyanin Ameliorates Colitis-Associated Colorectal Cancer by Regulating the Gut Microbiota and the IL-17 Signaling Pathway

Phycocyanin (PC) is a pigment-protein complex. It has been reported that PC exerts anti-colorectal cancer activities, although the underlying mechanism has not been fully elucidated. In the present study, azoxymethane (AOM)/dextran sulfate sodium (DSS)-induced mice were orally administrated with PC, followed by microbiota and transcriptomic analyses to investigate the effects of PC on colitis-associated cancer (CAC). Our results indicated that PC ameliorated AOM/DSS induced inflammation. PC treatment significantly reduced the number of colorectal tumors and inhibited proliferation of epithelial cell in CAC mice. Moreover, PC reduced the relative abundance of Firmicutes, Deferribacteres, Proteobacteria and Epsilonbacteraeota at phylum level. Transcriptomic analysis showed that the expression of genes involved in the intestinal barrier were altered upon PC administration, Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway analysis revealed the IL-17 signaling pathway was affected by PC treatment. The study demonstrated the protective therapeutic action of PC on CAC